Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Faster testing with reduced number of levels #130

Merged
merged 12 commits into from
Apr 9, 2024
Merged

Faster testing with reduced number of levels #130

merged 12 commits into from
Apr 9, 2024

Conversation

Melkiades
Copy link
Contributor

@Melkiades Melkiades commented Apr 2, 2024 •
edited
Loading

Ideally, we should be able to do tests with snapshot file sizes below 100Kbs. Before this I was getting stuck for tens of minutes on aet04 and I think we need this working faster and w/ lower memory footprint (it is impossible otherwise to review differences between versions). @zdz2101 can you help me track down if this is too much for the current test coverage?

The following reductions have been done in this PR:

Reducing levels of adae_pharmaverse for variable AEDECOD: keeping 15 levels out of 242 levels. Total kept (%): 46.8
Reducing levels of adlb_pharmaverse for variable PARAM: keeping 10 levels out of 49 levels. Total kept (%): 24.1

Fixes #127

@Melkiades Melkiades added enhancement New feature or request sme priority labels Apr 2, 2024
@Melkiades Melkiades requested a review from zdz2101 April 2, 2024 18:23
Melkiades
Melkiades commented Apr 2, 2024
View reviewed changes
tests/testthat/setup.R Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 3, 2024 •
edited
Loading

Unit Tests Summary

  1 files  111 suites   3m 4s ⏱️
245 tests 215 ✅  30 💤 0 ❌
480 runs  237 ✅ 243 💤 0 ❌

Results for commit b6adcce.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 3, 2024 •
edited
Loading

Unit Test Performance Difference

Test suite performance difference
Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
listing_lbl01 💚 $2.11$ $-1.60$ $0$ $0$ $0$ $0$
listing_lbl01_rls 💚 $2.11$ $-1.60$ $0$ $0$ $0$ $0$
listing_lbl02a_rls 💚 $3.24$ $-2.44$ $-1$ $0$ $0$ $0$
pagination_listing 💚 $9.37$ $-3.68$ $0$ $0$ $0$ $0$
pagination_table 💚 $25.85$ $-13.67$ $0$ $0$ $0$ $0$
table_aet01 💚 $3.51$ $-1.66$ $0$ $0$ $0$ $0$
table_aet01_aesi 💚 $5.22$ $-2.43$ $0$ $0$ $0$ $0$
table_aet02 💚 $128.76$ $-120.60$ $0$ $0$ $0$ $0$
table_aet03 💚 $37.53$ $-35.88$ $0$ $0$ $0$ $0$
table_aet04 💚 $372.37$ $-366.07$ $0$ $0$ $0$ $0$
table_aet04_pi 💚 $76.58$ $-69.56$ $0$ $0$ $0$ $0$
table_aet06 💚 $75.70$ $-71.97$ $0$ $0$ $0$ $0$
table_aet09 💚 $29.53$ $-27.56$ $0$ $0$ $0$ $0$
table_aet10 💚 $1.72$ $-1.59$ $0$ $0$ $0$ $0$
table_cmht01 💚 $3.29$ $-1.48$ $0$ $0$ $0$ $0$
table_cmt01 💚 $2.68$ $-1.17$ $0$ $0$ $0$ $0$
table_cmt01a 💚 $5.69$ $-2.60$ $0$ $0$ $0$ $0$
table_cmt01b 💚 $6.37$ $-2.93$ $0$ $0$ $0$ $0$
table_coxt01 💚 $3.22$ $-1.29$ $0$ $0$ $0$ $0$
table_disclosures 💚 $3.42$ $-1.73$ $0$ $0$ $0$ $0$
table_dmt01 💚 $4.97$ $-2.36$ $0$ $0$ $0$ $0$
table_dort01 💚 $2.77$ $-1.37$ $0$ $0$ $0$ $0$
table_dtht01 💚 $2.11$ $-1.06$ $0$ $0$ $0$ $0$
table_egt01 💚 $3.12$ $-1.58$ $0$ $0$ $0$ $0$
table_egt03 💚 $2.42$ $-1.20$ $0$ $0$ $0$ $0$
table_ent 💚 $28.52$ $-13.41$ $0$ $0$ $0$ $0$
table_ext01 💚 $3.10$ $-1.61$ $0$ $0$ $0$ $0$
table_lbt01 💚 $4.89$ $-2.91$ $0$ $0$ $0$ $0$
table_lbt02 💚 $2.73$ $-1.41$ $0$ $0$ $0$ $0$
table_lbt04 💚 $5.85$ $-4.60$ $0$ $0$ $0$ $0$
table_lbt05 💚 $37.07$ $-30.38$ $0$ $0$ $0$ $0$
table_lbt07 💚 $7.01$ $-4.61$ $0$ $0$ $0$ $0$
table_lbt10 💚 $2.53$ $-1.25$ $0$ $0$ $0$ $0$
table_lbt13 💚 $5.39$ $-2.98$ $0$ $0$ $0$ $0$
table_lbt14 💚 $175.11$ $-143.86$ $0$ $0$ $0$ $0$
table_lbt15 💚 $4.15$ $-2.68$ $0$ $0$ $0$ $0$
table_lgrt02 💚 $5.51$ $-2.30$ $0$ $0$ $0$ $0$
table_mht01 💚 $2.78$ $-1.36$ $0$ $0$ $0$ $0$
table_onct05 💚 $3.49$ $-1.76$ $0$ $0$ $0$ $0$
table_pkct01 💚 $2.36$ $-1.13$ $0$ $0$ $0$ $0$
table_pkpt03 💚 $2.24$ $-1.17$ $0$ $0$ $0$ $0$
table_pkpt05 💚 $2.02$ $-1.06$ $0$ $0$ $0$ $0$
table_rmpt06 💚 $2.85$ $-1.39$ $0$ $0$ $0$ $0$
table_rspt01 💚 $4.62$ $-2.37$ $0$ $0$ $0$ $0$
table_ttet01 💚 $6.48$ $-3.22$ $0$ $0$ $0$ $0$
Additional test case details
Test Suite $Status$ Time on main $±Time$ Test Case
listing_lbl01 💚 $2.11$ $-1.60$ LBL01_listing_is_produced_correctly
listing_lbl01_rls 💚 $2.11$ $-1.60$ LBL01_RLS_listing_is_produced_correctly
listing_lbl02a_rls 💚 $3.24$ $-2.44$ LBL02A_RLS_listing_is_produced_correctly
pagination_listing 💚 $4.33$ $-1.31$ Direct_pagination_works_fine
pagination_listing 💚 $5.04$ $-2.37$ pagination_counts_the_right_number_of_lines_when_wrapping_on_keycols_is_present
pagination_table 💚 $3.23$ $-1.52$ AET04_variant_2_page_by_pagination_tests
pagination_table 💚 $3.95$ $-2.10$ Direct_pagination_with_standard_values
pagination_table 💚 $3.04$ $-1.63$ Pagination_with_specific_column_widths_and_minimum_lines_per_page
pagination_table 💚 $10.95$ $-6.06$ Pagination_works_also_if_table_is_decorated
pagination_table 💚 $3.19$ $-1.61$ Pagination_works_for_page_types
table_aet02 💚 $5.53$ $-5.03$ AET02_variant_10_is_produced_correctly
table_aet02 💚 $6.75$ $-6.21$ AET02_variant_11_is_produced_correctly
table_aet02 💚 $5.23$ $-4.73$ AET02_variant_12_is_produced_correctly
table_aet02 💚 $8.11$ $-7.32$ AET02_variant_1_is_produced_correctly
table_aet02 💚 $9.16$ $-8.27$ AET02_variant_2_is_produced_correctly
table_aet02 💚 $38.35$ $-36.82$ AET02_variant_3_is_produced_correctly
table_aet02 💚 $3.56$ $-3.36$ AET02_variant_4_is_produced_correctly
table_aet02 💚 $5.78$ $-5.17$ AET02_variant_5_is_produced_correctly
table_aet02 💚 $5.32$ $-4.82$ AET02_variant_6_is_produced_correctly
table_aet02 💚 $29.93$ $-28.82$ AET02_variant_7_is_produced_correctly
table_aet02 💚 $5.37$ $-4.88$ AET02_variant_8_is_produced_correctly
table_aet02 💚 $5.67$ $-5.15$ AET02_variant_9_is_produced_correctly
table_aet03 💚 $37.53$ $-35.88$ AET03_variant_1_is_produced_correctly
table_aet04 💚 $12.08$ $-11.74$ AET04_variant_11_is_produced_correctly_with_Incidence_Rate_of_at_Least_X_all_SOCs_w_o_preferred_terms_removed_
table_aet04 💚 $70.26$ $-69.17$ AET04_variant_1_is_produced_correctly
table_aet04 💚 $41.17$ $-40.20$ AET04_variant_2_is_produced_correctly_Fill_in_of_Treatment_Groups_
table_aet04 💚 $81.92$ $-80.86$ AET04_variant_3_is_produced_correctly_Fill_in_of_Grades_
table_aet04 💚 $68.68$ $-67.62$ AET04_variant_4_is_produced_correctly_Collapsing_of_Grades_grades_1_2_grades_3_4_5_
table_aet04 💚 $73.01$ $-71.89$ AET04_variant_6_is_produced_correctly_with_an_Incidence_Rate_of_at_Least_5_totals_restricted_
table_aet04 💚 $13.36$ $-13.03$ AET04_variant_8_is_produced_correctly_with_an_Incidence_Rate_of_at_Least_X_Patients_
table_aet04 💚 $11.90$ $-11.57$ AET04_variant_9_is_produced_correctly_with_a_Difference_in_Incidence_Rate_of_at_Least_X_
table_aet04_pi 💚 $9.96$ $-9.13$ AET04_PI_full_table_is_produced_correctly
table_aet04_pi 💚 $8.23$ $-7.42$ AET04_PI_variant_1_is_produced_correctly
table_aet04_pi 💚 $11.77$ $-10.90$ AET04_PI_variant_2_is_produced_correctly
table_aet04_pi 💚 $8.49$ $-7.71$ AET04_PI_variant_3_is_produced_correctly
table_aet04_pi 💚 $7.76$ $-7.03$ AET04_PI_variant_4_is_produced_correctly
table_aet04_pi 💚 $8.07$ $-7.27$ AET04_PI_variant_5_is_produced_correctly
table_aet04_pi 💚 $9.84$ $-8.70$ AET04_PI_variant_6_is_produced_correctly
table_aet04_pi 💚 $9.68$ $-8.84$ AET04_PI_variant_7_is_produced_correctly
table_aet04_pi 💚 $2.79$ $-2.55$ AET04_PI_variant_8_is_produced_correctly
table_aet06 💚 $9.25$ $-8.46$ AET06_variant_1_is_produced_correctly
table_aet06 💚 $15.19$ $-13.92$ AET06_variant_3_is_produced_correctly
table_aet06 💚 $51.26$ $-49.59$ AET06_variant_5_is_produced_correctly
table_aet09 💚 $5.74$ $-5.02$ AET09_variant_1_is_produced_correctly_AE_related_to_study_drug
table_aet09 💚 $23.80$ $-22.54$ AET09_variant_2_is_produced_correctly_AE_related_to_study_drug_including_high_level_terms_
table_aet10 💚 $1.72$ $-1.59$ AET10_default_variant_is_produced_correctly
table_cmht01 💚 $2.41$ $-1.08$ CMHT01_variant_2_is_produced_correctly
table_cmt01b 💚 $2.46$ $-1.27$ CMT01B_variant_4_is_produced_correctly
table_egt01 💚 $3.12$ $-1.58$ EGT01_default_variant_is_produced_correctly
table_ent 💚 $8.93$ $-4.53$ ENT01_IT_is_produced_correctly
table_ent 💚 $5.54$ $-2.40$ ENT01a_IT_is_produced_correctly
table_ent 💚 $8.25$ $-3.83$ ENT02_IT_is_produced_correctly
table_ent 💚 $5.80$ $-2.65$ ENT02a_IT_is_produced_correctly
table_lbt01 💚 $4.89$ $-2.91$ LBT01_default_variant_is_produced_correctly
table_lbt02 💚 $2.73$ $-1.41$ LBT02_default_variant_is_produced_correctly
table_lbt04 💚 $5.85$ $-4.60$ LBT04_default_variant_is_produced_correctly
table_lbt05 💚 $13.86$ $-11.33$ LBT05_variant_1_is_produced_correctly
table_lbt05 💚 $12.15$ $-10.17$ LBT05_variant_2_is_produced_correctly
table_lbt05 💚 $11.07$ $-8.88$ LBT05_variant_4_is_produced_correctly
table_lbt07 💚 $7.01$ $-4.61$ LBT07_is_produced_correctly
table_lbt13 💚 $2.73$ $-1.86$ LBT13_variant_1_LOW_works_as_expected
table_lbt14 💚 $38.32$ $-31.83$ LBT14_variant_1_HIGH_works_as_expected
table_lbt14 💚 $36.28$ $-30.37$ LBT14_variant_2_LOW_works_as_expected
table_lbt14 💚 $11.89$ $-7.81$ LBT14_variant_3_LOW_without_baseline_missing_works_as_expected
table_lbt14 💚 $33.17$ $-27.48$ LBT14_variant_4_LOW_and_force_1_missing_both_baseline_and_post_baseline_then_force_the_missing_baseline_as_0_as_expected
table_lbt14 💚 $55.44$ $-46.36$ LBT14_variant_5_HIGH_with_fillings_works_as_expected
table_lbt15 💚 $2.20$ $-1.43$ LBT15_variant_1_works_as_expected
table_lbt15 💚 $1.95$ $-1.25$ LBT15_variant_2_works_as_expected
table_pkpt03 💚 $2.25$ $-1.17$ PKPT03_Drug_X_is_produced_correctly

Results for commit 62d28e7

♻️ This comment has been updated with latest results.

@Melkiades
Copy link
Contributor Author

from checks:

Found the following significant warnings:
  Warning: replacing previous import ‘formatters::var_relabel’ by ‘random.cdisc.data::var_relabel’ when loading ‘scda.test’

We need to fix #129 too here

@Melkiades Melkiades linked an issue Apr 3, 2024 that may be closed by this pull request
Decide to use only {formatters} import now that has been fixed #129
Closed
zdz2101
zdz2101 reviewed Apr 3, 2024
View reviewed changes
Copy link
Collaborator

@zdz2101 zdz2101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With regard to the pharmaverseadam stuff, looks great! See my other comments as well. Also wondering why the 10 new "<- NULL" error stuff is showing up, they don't show up when I'm testing each file individually but it shows up when I click the "test" button on the top right of the rstudio ui

tests/testthat/setup.R Show resolved Hide resolved
tests/testthat/test-table_ratet01.R Show resolved Hide resolved
tests/testthat/_snaps/table_ratet01.md Show resolved Hide resolved
@Melkiades
Copy link
Contributor Author

With regard to the pharmaverseadam stuff, looks great! See my other comments as well. Also wondering why the 10 new "<- NULL" error stuff is showing up, they don't show up when I'm testing each file individually but it shows up when I click the "test" button on the top right of the rstudio ui

I have no idea! I will investigate ;)

@Melkiades
Copy link
Contributor Author

Melkiades commented Apr 4, 2024 •
edited
Loading

@zdz2101 @edelarua @shajoezhu, now the tests should all work. Testing time reduced to < 10min (it says 256s) and snap file size reduced to <=100 KB (one was 1.4Mb before) ;)

@Melkiades Melkiades mentioned this pull request Apr 9, 2024
Open
zdz2101
zdz2101 approved these changes Apr 9, 2024
View reviewed changes
Copy link
Collaborator

@zdz2101 zdz2101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Melkiades Melkiades merged commit 83fc384 into main Apr 9, 2024
24 checks passed
@Melkiades Melkiades deleted the faster_testing_and_fixes@main branch April 9, 2024 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edelarua edelarua edelarua left review comments

@zdz2101 zdz2101 zdz2101 approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request priority sme
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Decide to use only {formatters} import now that has been fixed Faster testing with pre-pruning
3 participants
@Melkiades @zdz2101 @edelarua